Papers with supervision strategy
Layer-wise Fusion with Modality Independence Modeling for Multi-modal Emotion Recognition (2023.acl-long)
Copied to clipboard
Jun Sun, Shoukang Han, Yu-Ping Ruan, Xiaoning Zhang, Shu-Kai Zheng, Yulong Liu, Yuxin Huang, Taihao Li
| Challenge: | Existing studies focus on developing models that exploit the unification of multiple modalities. |
| Approach: | They propose to maintain modality independence by using a multi-modal transformer model that fuses all modalities. |
| Outcome: | The proposed model outperforms state-of-the-art models in multi-modal emotion recognition. |
Deal, or no deal (or who knows)? Forecasting Uncertainty in Conversations using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Effective interlocutors account for the uncertain goals, beliefs, and emotions of others. |
| Approach: | They propose to calibrate language models to better represent outcome uncertainty . they propose to use two methods to calibrated small open-source models . |
| Outcome: | The proposed fine-tuning strategies can calibrate smaller open-source models to beat pre-trained models 10x their size. |
Beyond Transcription: Unified Audio Schema for Perception-Aware AudioLLMs (2026.findings-acl)
Copied to clipboard
Linhao Zhang, Yuhan Song, Aiwei Liu, Chuhan Wu, Sijun Zhang, Wei Jia, Yuan Liu, Houfeng Wang, Zhou Xiao
| Challenge: | Recent Audio Large Language Models (AudioLLMs) excel at reasoning tasks, but struggle at elementary auditory perception. |
| Approach: | They propose a framework that organizes audio information into three explicit components in a unified JSON format. |
| Outcome: | The proposed framework boosts fine-grained perception by 10.9% on MMSU over state-of-the-art models while preserving robust reasoning capabilities. |